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Abstract 

We report on the restoration of gray-scale image when it is decomposed 
into a binary form before transmission. We assume that a gray-scale image 
expressed by a set of Q-Ising spins is first decomposed into an expression 
using Ising (binary) spins by means of the threshold division, namely, we 
produce {Q — 1) binary Ising spins from a Q-Ising spin by the function 
F(o~i — m) = 1 if the input data o~i £ {0, • • • , Q — 1} is o~i > m and 
otherwise, where m £ {1, ■ • • , Q — 1} is the threshold value. The effects of 
noise are different from the case where the raw Q-Ising values are sent. We 
investigate which is more effective to use the binary data for transmission or 
to send the raw Q-Ising values. By using the mean-filed model, we analyze 
the performance of our method quantitatively. In order to investigate what 
kind of original picture is efficiently restored by our method, the standard 
image in two dimensions is simulated by the mean-field annealing, and 
we compare the performance of our method with that using the Q-Ising 



form. We show that our method is more efficient than the one using the 
Q-Ising form when the original picture has large parts in which the nearest 
neighboring pixels take close values. 
PACS numbers : 02.50.-r, 05.20.-y, 05.50.-q 
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I. INTRODUCTION 



Statistical mechanical approaches to the problems in information science are employed 
frequently, and it is now known that such methods are very useful in various problems 
Among such problems, the image restoration problem has been investigated both 
theoretically and practically We usually send and receive the information by means 

of various networks. The information is the data of document, sound, picture, and so on. 
However, it is generally impossible for a receiver to receive complete transmitted data 
when a sender transmits something, because the data are transmitted through a noisy 
channel. If we restrict the type of data to that of the picture, the original image is affected 
by some kind of noise when it is sent by a defective fax, a fickle e-mail, etc. When we 
receive such a corrupted image, we have to convert it using some kind of filter to obtain 
the original image. Basically it is the image restoration problem to estimate the original 
data (the original image) from the received, corrupted data (the degraded image). We 
may regard the digital picture as a discrete spin system. For example, a black and white 
image (WBI) corresponds to an Ising spin system by identifying the white color with +1 
and the black with —1. Furthermore, the theory of image restoration is constructed by 
considering that two axes of the plane, x-axis and y-axis, correspond to two axes of time 
in the stochastic process. That is, the Markov process that the event occurs at a specific 
time is affected by what happened at neighboring pixels at a one-time step before. 

There are mainly two standard approaches to the image restoration by means of the 
method of statistical mechanics. One is called the maximum a posteriori (MAP) esti- 
mation in which the estimation of the original image is given by maximizing a posterior 
probability distribution. This estimation will be seen to correspond to a search of the 
ground state of spin system described by the effective Hamiltonian in the context of sta- 
tistical mechanics. Another is the estimation in which we regard the expectation value 
with respect to the maximized marginal posterior probability at each site in thermal 
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equilibrium as the original image and is called the maximum posteriori marginal (MPM) 
estimation. This estimation is also called the finite temperature restoration. Therefore, 
the MPM estimation includes the MAP estimation. Among the two estimations, it was 
proposed by Marroquin et al || that the MPM estimation gives better performance than 
the MAP estimation. Nishimori and Wong || proved this fact for black and white images 
using a rigorous inequality. With respect to the gray-scale image, the same has been 
shown by Tanaka J7| from a different viewpoint. 

Not only black and white images but also gray scale images are actively investigated 
by many people in the field of statistical mechanics. Restoration of gray scale image using 
chiral Potts spin || may not be appropriate since chiral Potts spin can not express the 
distance among different states. However Q-Ising spin || can express the gray scale level 
at least. Restoration of gray scale image using Q-Ising spin was first investigated by Inoue 
and Carlucci fllQfl . 

Inspired by their studies, we investigate the restoration of the gray scale image ex- 
pressed by the Q-Ising spin when it is decomposed into the Ising spin before transmission. 
In this method, the gray scale image is decomposed into binary data by means of the 
method of threshold division before it is transmitted. Then the bit-decomposed data is 
transmitted through a noisy channel. Therefore the effects of noise is different from those 
in the Q-Ising form. We show how well the original image is restored in our method in 
comparison to the restoration using the Q-Ising form. 

This paper is organized as follows. In the next section, we explain the general formu- 
lation of image restoration and the method of threshold division. In Sec. Ill, we analyze 
the static properties of image restoration in the infinite-range model. In Sec. IV, we 
verify the result of the infinite-range model in realistic pictures by means of Monte Carlo 
simulation. In Sec. V, standard image is restored by the method of mean-field annealing. 
The performance of restoration in our method is compared with that using the Q-Ising 
form. The final section is denoted to summary and discussions. 
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II. GENERAL FORMULATION 



A. Image restoration 

A gray scale image is represented by a set {^}i=i,...,jv of fixed values. The variable & 
takes an integer value between and Q — l. Even in black and white image in addition to 
gray scale image, most of natural images have nontrivial structures generally. Therefore, it 
is impossible to discuss the property of a given specific natural image exactly. Nevertheless 
we may notice that an eminent property of natural images is local smoothness. Therefore 
we assume that the original image is generated by the Boltzmann probability represented 
by the following: 



-§E(fc-fc) s 



(i) 



where (ij) represents interacting sites and z is the coordination number. Z(f3 s ) is the 
normalization constant, and (3 S (= T" 1 ) is the inverse temperature to generate the original 
image. If f3 s is large, an original image with many clusters consisting of the same value is 
generated. 

A degraded image is generated by sending data of the original image through a noisy 
channel. We consider two kinds of noise. One is the binary noise caused by a binary 
symmetric channel (BSC) and another is the Gaussian noise caused by the Gaussian 
channel (GC). 



1. Gaussian channel 



In the Gaussian channel, the output 7$ for an input £j is a Gaussian random variable 
with mean r £j and variance r 2 . The probability distribution of output given the input 
{£} is written as 



P({r}m) = 



2-kt 



cxp 



2r 2 



(2) 



The degraded image {r}j =1 ... )A r is generated by this probability distribution. 

According to the Bayes formula, the posterior probability that the es- 

timate of source sequence, namely the restored image, is {a} i=1 ... N , provided that the 
output is {r}, is given as 

P(frrMM) f (HIW)W) 
11 111 iJ tr w P({r}|W) J R m ({a}) 

~ exp ( - hJ2(°i - n) 2 _ a i) 2 ) 

\ i ij / 

= exp(-W G ). (3) 

Since we can use the degraded image only and do not know the other information of 
original image, we introduced the model prior P m ({a}) to represent a priori knowledge 
on natural image: -P s ({£}) represented by equation (]!]). Furthermore prior parameters (3 S 
and To/r 2 to control the generation of original image and degraded image, respectively, 
are unknown quantities. Accordingly, we have to use the so-called hyper-parameters f3 m 
and h instead of prior parameters f3 s and Tq/t 2 . Then, controlling these hyper-parameters, 
we can obtain the optimal restored image. 



2. Binary symmetric channel 



When binary data {£&} (Q = 2) takes or 1, the type of noise can also be binary. 
In such pixel of the original image is flipped with the probability p, the error 

rate. The error probabilities of flipping the signal +1 to and to +1 are the same. The 
probability distribution of this output {t&} G {0, 1} is expressed as follows. 



P{{n}\{&}) = ^exp 



(4) 



where 



Z T = Tr T6 exp 



i 

N 
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The parameter j3 T is defined by p becomes 



(3 T = \n^. (5) 
p 



The posterior probability in this case is 

P({n}\W b })P m (W b }) 



P(W b }\{n}) 



tr ab P({n}\{a b })P m (W b }) 
exp ^ - h EK?> ~ T i, b ) 2 
exp(-W B ). (6) 



B. MPM estimation and mean square error 

Next, we discuss the method to estimate the original image from the preceding poste- 
rior probability distribution and to evaluate the restored image which is obtained by such 
estimations. 

We consider the marginal distribution obtained from the posterior probability distri- 
bution equation (|3|) (or (|6|)) to estimate the original image: 

PWtr}) = E P({<7}\{T}) (7) 

Using this marginal probability distribution, we calculate the local magnetization at a 
site % as given by 

Q-i 

(n)fUfi= E^(^iW)- (8) 

<7i=0 

Since the above expectation value is a continuous real number, we need to change it into 
an integer by means of the following function Q: because the original image consists of 
the discrete value in digital images, the restored image is the estimation of original image 
should, properly, take the discrete value. 

2Jfe-l\ , v 2k + V 



k=l 



®((Vi}pm,h g ) - Q (( <T i)pfn,h 2^—) 



(9) 



A real number (cr^)^^ is translated into the closest integer. Then we regard the above 
discrete value f2((o"j)) as the value of i-th pixel for restored image at finite (3 m and h in 
the finite temperature process. 

In the MAP estimation, the estimation of the original image is regarded as the set {a{\ 
which maximizes the posterior probability distribution (Eq.s (0),©). In other words, it 
is the set that minimizes the energy of the system described by the Hamiltonian H,q or 
TCb and is actually the ground state. Consequently, the above estimation corresponds to 
the MAP estimation when T m —>■ (f3 m — > oo) keeping H = h//3 m constant. 

It is very important to evaluate the performance of restoration by means of these 
estimations. For this purpose, we assume that we know the original image and evaluate 
the performance of restoration by measuring the distance between the original and the 
restored images given by the above estimation. In order to measure the distance between 
the original and the restored images, we use the following mean square error as the 
distance, 

^d = ^E^-^(^W)] 2 ; (io) 

i 

whose value depends on the hyper-parameters h and f3 m appearing in the thermal average. 

C. Method of threshold division 

We next discuss the restoration of gray-scale image using bit-decomposed data. It is 
expected that the effects of noise on the binary data is lower than that on the Q-value 
data for image restoration: the binary data is estimated easily compared with the Q-value 
data because of the explicit representation. Since the original image is estimated by using 
the information of degraded image, the performance of restoration ought to be deeply 
affected by the effects of noise. Therefore we expect that the performance of restoration 
is improved by using the binary data instead of the Q-value data. 
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We generate binary data from Q-value data using the Q-Ising form by means of the 
following function 

e& - fc) = = i (&>ao, o^<k) (ii) 

where k (G {1, 2, • • • , Q — 1}) is the threshold value and £j (e {0, 1, • • • , Q — 1}) is the input 
data. We obtain [Q — 1) sets of binary data from a Q- value data by using this function 
with threshold value changed from 1 to Q— 1. This method is called the threshold division. 
For example, we consider a set with Q = 3 (0, or 1, or 2) and six pixels: {1, 2, 1, 1, 0, 2}. 
If we insert this set into the above function with the threshold value k = 2, the binary 
set {0, 1, 0, 0, 0, 1} is generated. The set {1, 1, 1, 1, 0, 1} is also generated when k = 1. 
Thus two binary sets {0, 1, 0, 0, 0, 1} and {1, 1, 1, 1, 0, 1} are generated from the 3-value 
set {1,2,1,1,0,2}. These operations are depicted in FIG. [I]. The expression of binary 
data £i,A;(G {0, 1}) produced by this method is different from the usual binary notation 
because the relation between a Q- value data and (Q — 1) sets of binary data becomes 

Q-l 

& = 6,1 + 6,2 + ■ • • + ti,Q-l = E (12) 

fc=l 

We call binary data generated by this threshold division the bit- decomposed data (BDD). 

Next, the process in the restoration of gray-scale image using the bit-decomposed data 
becomes the following. 

1. Decompose a Q- value data (original image) into (Q — 1) sets of binary data before 
transmission. 

2. Send (Q — 1) sets of binary data through a noisy channel. 

3. Receive (Q — 1) sets of corrupted binary data (degraded image). 

4. Restore the original image from the degraded image. 

Obviously, this procedure is different from the method in which the raw Q-value data are 
sent. 
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Following the general formulation mentioned in the previous section, we formulate the 
above process. The posterior probability necessary for restoration is the following 

P{Wmn\, |T- 2 }, • ■ ■ {TQ-i}) = p/r -, r -, 7- HJ^p (S„\\ 

tr {a} P({n}, {t 2 \, ■ ■ ■ {TQ-i}\{a})P m {{a}) 
~ ex P ^SSK*: - T i,kf ~ ^£(<*i - Vjfj > ( 13 ) 

where we used {a} to denote the dynamical variables used to estimate {£}. The notation 
0i,fc ^ {0, 1} is the estimation of decomposed data ^fc. The notation G {0, 1} is the 
degraded binary data. The relation between er^, and (Tj, Tj is the same as that between 
£iand&,* (Eq. ©). 

In the restoration of gray-scale image using the Q-Ising form, the posterior probability 
distribution is given by 

P(W|{r}) ~ exp i-h^i ~ n) 2 - ^ " *i) 2 j • (14) 



The difference of the posterior probability between our method ( |T3| ) and the Q-Ising form 
(P^) is only in the random field term. That is to say, the effects of noise in our method 
are different from the case where the raw Q-Ising data are sent. 



III. ANALYSIS OF THE INFINITE-RANGE MODEL 

In this section, we discuss the restoration of the gray-scale image in the infinite-range 
model (mean- field model). Since the infinite-range model is the model in which all pixels 
interact mutually, it is not useful for restoration of real images directly. However, the 
analysis of image restoration in the infinite-range model is very useful in understanding 
qualitatively the property of macroscopic quantities. For this reason, we investigate the 
averaged performance of our method by using the infinite range model. 

Suppose that the prior probability generates the original image in the infinite-range 
model like Eq. ([]]). Then the z-th pixel interacts all other pixels and the prior probability 
distribution is represented as 
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zip. 



-exp 



2N 



1.1 



(15) 



where N is the system size and Z(/3 S ) is the partition function of the ferromagnetic Q-Ising 
model. We assume that the original image is generated by this probability both in the 
bit-decomposed data case and the Q-Ising case. In this section we adopt as the original 
image a snap shot of the system produced by the above probability. 

We treat the Gaussian channel for simplicity. In our method, the Gaussian channel in 
which input data are expressed in a binary form is given by 



Pk{{Ti,k}\{Zi,k}) 



2ttt 



cxp 



2r< 



(16) 



where Tj f. is a Gaussian random variable with mean tq^ & and variance r 2 . The input £j 



takes or 1. For comparison, the Q-Ising case is denoted 



P({n}\{ti}) 



l 



2vrr 



; exp 



71 £fa-T&) 2 



2r 



(17) 



where £j takes a value between and Q — 1, and Tj is a Gaussian random variable with 
mean r^j and variance r' 2 . We calculate the posterior probability using Eq. fllfiD and Eq. 

(0)- 

In the bit-decomposed data case; 



P{{*}\ in}, {r 2 }, ■ ■ ■ {tq-x}) ~ II Pk({r k }\W k })P m ({a}) 



~ exp I -/i^^Kfc - r J>fc ) 2 - ^ ^(cri - cx,) 2 
= exp (-H eS ) . 



In the Q-Ising case; 

P({a}|{r})~ exp I -r.) 



2 /^m V~V >2 



(19) 



The difference of these expressions and equations (|13|) , dl4|) in previous section is only 
that the coordination number z became N. 



9 



We calculate the free energy from these probability distributions to clarify the behavior 
of macroscopic quantities. Since there is randomness in the field for the spin system 
described by the effective Hamiltonian if eff in equations ([18]) and (|IDD , the free energy 
must be averaged over the probability distribution of the random field in addition to the 
thermal average. Accordingly, we calculate the free energy per site by 

/ = ^ /7V = __L_ [lnZ] . (20) 

By using the replica method, we obtain the following replicated partition function, 
[Z n \ =J2Y,Psm)P({r}m)Tr«e- H ^ 

xTY.expl-h^SEK* -<*) 2 " " °i) 2 ]. ( 21 ) 

where a is the label of n dummy replicas. Assuming replica symmetric ansatz m a = 
m (Va), the following expressions of the order parameters are derived. 

Id] = mo = i^T'^^ 2 ™" 34 "' 3 * 5 '. < 22 ) 

^ = m = W^ 2 '""" 4 "^ /"»■•••/ < 23 > 

z lPsJ •/ •/ tr CT exp(i7 cfr ) 

where we defined = (dii/\/27r)e _M2 / 2 and 

# e ff = 2p m ma - (3 m o 2 -h^a 2 k + 2hr ^ u k a k + 2/ir ^ ( 24 ) 

k k k 



Equation ( |22| ) represents the magnetization of the original image in the infinite range 
model. The behavior of the source magnetization mo as a function of the temperature j3 s 
is shown in FIG. H. Equation (^) is the equation of state for the magnetization of the 
restored image. In the limit N oo, the mean square error (Eq. (|10|)) is rewritten as 

#d = zTg)^^ ^'^ J Du ^--J DuQ-dti ~ ^(«>)} 2 - (25) 
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We next assume a condition to compare the performance of restoration between using 
the bit-decomposed data and the Q-Ising form. The condition is that the distance between 
the original and the degraded images in our method is equal to that in the Q-Ising 
form. We can compare the difference of performance in disparate noisy channels by 
this condition. The distance between the original and degraded images in our method 
-^(BDD) is expressed similarly to the above expression. It turns out that the distance 
ifp(BDD) becomes simple in the thermodynamic limit N — > oo. 

#5(BDD) = ^ + 6,2 + ■ ■ • + &,Q-i) - fai + Ti,z + ■ ■ ■ + r liQ -i)} 2 



k k 1 n»-,7-Q-i»£i--£Q- 



(Q - l)r 2 + 
(Q - l)r 2 . 



TO 



" tr 6-5Q-i 



fc=i 



(26) 



When the mean To of the Gaussian noise is 1, the distance ifp(BDD) depends on the 
variance r 2 only. In the Q-Ising form, i?g(Q-Ising) is 



F5(Q-Ising) = -5:{6-r l } 2 



' 2 i 
r + 



K - K 



-tr, 



2/3 s m £-/3 3 £ 2 



_/2 



r 



(27) 



where r' 2 is the variance of a Gaussian channel in the Q-Ising form and is different 
from t in the bit-decomposed data. When Tq = 0, H[ ) (Q-lsmg) = r' 2 similarly to the 
bit-decomposed data case. As the rate of degradation is equal to each other, namely 
iJg(BDD) = iJg(Q-Ising), the variance of a noisy channel between in our method fl2"E| ) 
and that in the Q-Ising form ([27]) satisfies the following relation. 

(28) 



Next, we calculate the restoration of gray-scale image when Q — 3. We plot the 
magnetization m of original image as a function of source temperature T s for Q = 3 
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in FIG. [| In high temperature region T s — > oo, the magnetization becomes m = 
(0 + 1 + 2)/3 = 1 since each spin takes all the values with the same probability 1/3. 
We see that two locally stable states are generated in the middle range of temperature. 
The globally stable state is the line of uiq = 1. The transition temperature between the 
paramagnetic phase and the ferromagnetic phase is T c ~ 0.9. These locally stable states 
become more stable with the decrease of temperature and correspond to the globally 
stable states m = and m = 2 in T s = 0. That is, the system of the ferromagnetic 
Q-Ising model for Q = 3 has triple degeneracy. 

We use a snapshot of the system when the magnetization is mo = 1 at T s — 0.75 as the 
original image. When the distance between the original image and the degraded image is 

= 1.0, macroscopic quantities behave as in FIG. |3|. The magnetization m of equation 
(p3|) at the ratio of hyper-parameters H = h//3 m = 0.25 has three stable states at T m = 
as mo we see in the original image. At low temperature, the system has one globally 
stable state m = 1 and two locally stable states m = and m = 2. However these locally 
stable states vanish as the ratio of hyper-parameters H becomes large. Seeing the free 
energy in FIG. [3], we find that the state of system branches out into a global minimum 
and local minima at T m = 0.27. 

The mean square error using m of the global minimum is smaller than that using 
m of local minimum in all temperature region and gives the optimal value for the ratio 
of hyper-parameters H. The mean square error Hd between the m of global minimum 
and the original image m in some ratio of hyper-parameters H are described in FIG. |j. 
The smallest mean square error is given at T m = 0.75 and H = 0.75. When the distance 
between the original and degraded images is 1.0 (H^ = 1.0), we see that the deviation 
r of Gaussian noise becomes 1/y/Q — 1 = 1/V2 by equation (p6|). Therefore, when the 
ratio of hyper-parameters corresponds to that of the prior parameters (H = h//3 m = 
(t /2t 2 ) / fi s = 0.75), the optimal restoration is given at the temperature corresponding 
to the source temperature (T m = T s = 0.75) similarly to the Ising model. At high 
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temperatures T T 



711 



oo, all states appear with equal probability. Then the thermal 



average of a spin a becomes (a) = (0 + 1 + 2)/3 = 1 at all pixels. The mean square error 
is 



If the distance between the original and degraded images, iJg(BDD) or ifg(Q-Ising), is 
extremely large, the performance of restoration does not become smaller than this value. 

We compare the optimal performance of restoration using our method with that using 
the Q-Ising form under the condition that the degradation of image in our method is 
equivalent to the one in the Q-lsing form, namely (BDD) = if J (Q-Ising form). In 
the Q-Ising form case, the deviation is r = 1.0 by equation (|27| ) when = 1.0. The 
optimal restoration in the Q-Ising form is also given in the ratio of hyper-parameters 
corresponding to that of source parameters (H = h/j3 m = (r /2r 2 )//9 m = 0.375). The 
optimal performance of restoration in both our method and the Q-Ising form is shown in 
FIG. U We see that the restoration by our method gives better performance than that of 
the Q-Ising form. 



In this section, we investigate realistic pictures in two dimensions and check the results 
obtained in the infinite-range model. It is, however, difficult to investigate the restoration 
of real images by analytical methods. We discuss the restoration of two-dimensional 
images by means of Monte Carlo simulations. 

We suppose that the original image is generated by the Boltzmann probability distri- 
bution as we did in the infinite-range model case and we use a snap shot of the QTsmg 
model at a temperature as the original image. 

In digital images, the pixel takes discrete values. Even if real-valued Gaussian noise 
is added in the degradation process, we have to regard such a noise as the discrete noise 




(29) 



IV. MONTE CARLO SIMULATION 
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(for example, binary noise) when both the original and the degraded images are digital. 



(This is essentially the same situation as in a error- correcting codes. |TI]) Therefore, 
we consider the binary noise caused by the binary symmetric channel with the error 
probabilities p = 0.10 (Q = 4) and p = 0.15 (Q — 8), which correspond to the parameters 
(3 T ~ 2.2 and (3 T ~ 1.7 by Eq. (||). The size of digital image is 100 x 100 and averages 
over ten samples are taken at each data point. 

FIG. || (the upper figure) represents the mean square error of several H when Q = 
4, T s = 0.35, m ~ 1.2, p = 0.10 and ~ 0.30. The optimal restoration is given at 
T m — T s , H = h/p m = /3 T /Ps ~ 0.75 like in the infinite-range model. 

The mean square errors for Q — 8, T s = 0.4, (0.6), m ~ 2.2, (3.0), p = 0.15, ~ 
1.0 are also shown in FIG. [5] (the lower two figures.). We see that the best performance 
of restoration at each temperature is obtained at the ratio of hyper-parameters H corre- 
sponding to that of the prior parameters, and the performance of restoration in T s = 0.4 
(the lower left) is better than that in T s = 0.6 (the lower right) as written before. 



V. MEAN-FIELD ANNEALING 

The amount of computation required for restoration by means of Monte Carlo sim- 
ulation is enormous. Hence, in the present section, we reconstruct the original image 
from binary degraded images by means of the mean-field annealing [|l2| , |l"3f with periodic 
boundary conditions. This method enables us to search for the optimal solution quickly. 
We apply the mean-field approximation to the posterior probability distribution. 

P(WI{l},W,-{l!-i}) 

= ziih) exp {~ h ^¥"- k " %tf " ^V"' " a,f ) (30) 

In order to treat each site separately, we trace out with respect to all pixels besides i-th 
pixel in the above probability distribution. 
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Pij( n )= II tr ^i,-,<XQ-i P ({°'}l{ T l}>{ T 2},---{TQ_l}) 

kl^ij 

= Tr ait ... j(TQ _ 1 P({cr}\{T 1 }, {r 2 }, • • ■ {r Q _i})5(n, a^) (31) 

where (ij) represents the site index in two dimensions: % and j are the coordinates of 
x and y axes, respectively. This expression is called the marginal probability distribu- 
tion. According to mean-field approximation, the posterior probability distribution (|30|) 
is approximated by the product of the marginal probability distribution ([H]) as follows. 

P({*}\{ti}, {r 2 }, • • • {t q _ x }) ~ J] (32) 

The recursion relation for the iterative algorithm is derived from a variational principle. 
To derive the recursion relation, we substitute the expression of approximation (|3"2| ) to 
the free energy. 

F(p) = E(p)-T m S(p). (33) 

The variation of the above free energy is calculated with respect to the marginal proba- 
bility distribution at a certain site: pij. Finally, the recursion relation with respect to the 
local magnetization for the iteration algorithm is obtained as 

t+ l Eo-yio ex P(#MF) / q yi \ 

rriij = 3 - , (34) 

^MF = ^( m li+l + m i,j-l + m \+l,j + 

(35) 

k k 

where Zmf is the normalization constant in the mean-field approximation. 
We solve the above relations numerically under the convergence condition 

i N 

& = ^ y L\™t 1) -< ) \< 1 °-*> ( 36 ) 

and obtain approximately the restored image at each temperature. The annealing schedule 
is set at AT m = 0.01. 
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We compare the performance of restoration using our method with that using the Q- 



Ising form for five kinds of standard images [O]. We choose Q = 8, system size=200 x 200 
and i?J ~ 1.00. The result is shown in table |. The original versions of these five standard 
image are shown in FIG. 

The original, degraded and optimal restored images in our method and Q-Ising form 
are shown in FIG. [7| for the case of "lena". 

In TABLE D, the ratio of pixels at which the nearest neighboring pixels take the 
same value is also represented. We see that the performance of restoration using our 
method is better than that using the Q-Ising form in the standard pictures "chair" and 
"girl". However, the QTsing form is better in "house", "lena" and "mandrill". One of 
the differences is that "chair" and "girl" have a lot of large parts and do not have short 
edges compared with the other pictures as we see intuitively. In TABLE [I], that "chair" 
has obviously large parts is quantitatively expressed by the ratio of pixels at which the 
nearest neighboring pixels take the same value. However, the ratio in "girl" is as much 
as that in "house" and "lena". 

In order to investigate this aspect in more detail, we show the ratio of pixels at which 
the difference among the value of the nearest neighboring pixels is smaller than 2 in table 
|H[ As seen in TABLE [0], "girl" is similar to "chair" rather than "house" and "lena" . The 
effects of noise in the bit-decomposed data and Q-Ising form are clarified by comparing the 
NNP1-D in TABLE | and NNP2-D in TABLE 0. NNP1-D in our method is smaller 
than that in the Q-Ising form for all kinds of standard images, but the decrease of NNP2 
from the original image to the degraded image in our method is small compared with that 
in the Q-Ising form for five standard images. That is, the noise in our method affects 
the original image widely, but the shift of value in a site is small. On the other hand the 
shift of value in a site is large in the Q-Ising form. Therefore the noise in our method 
has the effect of smoothing, and the restoration using our method is efficient for original 
images in which eliminating noise is more significant rather than preserving informations 
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of original image as "chair" and "girl" . 

Furthermore, we compare the performance using our method and in the Q-Ising form 
for the ifg ~ 2.0 case. Other conditions are the same as the previous case. We can simply 
investigate the tolerance of our method and Q-Ising form against noise by comparing 
~ 1.0 and 2.0 cases. The performance of restoration is shown in TABLE [Ell] and 
the resultant image is presented in FIG. || for the case of "lena". The performance 
of restoration is clearly worse than that in the previous case for all pictures, and the 
difference of performance between our method and the Q-Ising form is clear. Moreover, 
the restoration using our method gives better performance than that using the QTsing 
form in standard image "house". The result is different from that when ~ 1.0. In the 
Q-Ising form, large clusters remain in the restored image as in FIG. ||. Accordingly we 
conclude that the restoration using the bit-decomposed data is not affected by the noise 
compared with that using the Q-Ising form in this case. 

Consequently we found that the restoration using our method is more efficient than 
that in the Q-Ising form in original images where the nearest neighboring pixels take the 
close values. The reason is that to eliminate the added noise is more significant than the 
information of the original image in such images. Regardless of the Q-Ising form and our 
method, the restoration of such images tend to give good performance compared with 
that of other images in the image restoration using the method of statistical mechanics. 
Moreover, our method can give the adequate restored image in original images "chair", 
"girl" and "house" even if the noise is strong to a certain extent: for example, ~ 2.0. 

Finally, we try to restore the original image by using the composition of our method 
and the Q-Ising form. Two processes are considered in restoration using the composition. 
One is the method of restoration where the original image is restored from Q — 1 sets of 
binary degraded images which are generated from the received, degraded Q-value image 
by the method of threshold division. Another is the method of restoration where the 
original image is restored by using the QTsing form after translating Q — 1 sets of binary 
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degraded images into a Q-value degraded image. These restoration processes are shown 
in FIG. g. 

The second process is not efficient in the restoration of gray-scale image because the 
performance of restoration is worse than that both using our method and the Q-Ising 
form. On the other hand, the first process gives interesting results as in TABLE |IV| when 



if g ~ 1.0. Seeing in TABLE [TV], we find that this method gives better performance than 
our method and the Q-Ising form for all standard images: "chair" , "girl" , "house" , "lena" 
and "mandrill". The restoration of standard images using this method for the case of 
"lena" is shown in FIG. [K| This idea that, after a Q- value data are received, they are 
decomposed into binary data has been already used in the field of engineering. The filter 
is called the Stack filter |Tj| which works on decomposed binary data after reception. 

The Stack filter by means of the method of statistical mechanics as the first process 
gives better performance than the other two methods, but the theoretical framework is 
not clear. Because we have to assume the noisy process matching the degraded data in 
the Bayesian viewpoint, it is very difficult for us to estimate the noisy channel matching 
such binary data. Accordingly, it is well possible that the hyper-parameter h can not be 
estimated appropriately for the hyper-parameters estimation in the Stack filter by means 
of the method of statistical mechanics. 



VI. SUMMARY AND DISCUSSIONS 

In this paper, we investigated the restoration of gray-scale image using the bit- 
decomposed data instead of the conventional Q-Ising form and found the conditions that 
our method is more efficient than the method of the Q-Ising form. 

In the infinite-range model, we analyzed the image restoration when the original image 
is affected by the Gaussian noise, and obtained the static properties of image restoration. 
In order to obtain the averaged performance, we calculated the restoration of gray-scale 
image by means of the replica method. Then we found that the mean square error has 
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a minimum at the finite temperature for any ratio of the hyper-parameters H . However, 
when the noise rate is extremely large, the minimum sinks under the mean square error 
Hr>(T m — > oo) in the high temperature limit. In other words, we can only obtain the 
restored image in which all states appear with equal probability as the optimal restored 
image when the original image is very corrupted. The best performance of restoration 
is given at the temperature at which the original image is generated, when the ratio 
of hyper-parameters corresponds to that of the prior parameters. This result is also 
common between the Ising spin and the Q-Ising spin cases. Furthermore, we obtained the 
quantitative result that the performance of restoration using our method is better than 
that using the Q-Ising form when /Jg(BDD) = H^(Q- Ising). 

We analyzed the restoration of realistic images in two dimensions by means of Monte 
Carlo method and the mean-field annealing approximately. Using Monte Carlo method, 
we confirmed the result of the infinite-range model that the mean square error gives 
the optimal minimum value at the temperature corresponding to the source temperature 
when the ratio of hyper-parameters corresponds to prior parameters for a snap-shot of 
the Q-Ising model for Q = 4 and Q — 8. We restored five standard images by means 
of the mean-field annealing and found that the restoration using our method is better 
than that using the Q-Ising form in the original images where keeping the information of 
original image is not significant compared with the eliminating the added noise as "chair" 
and "girl". The reason is that the effects of noise in our method contain the effect of 
smoothing. However, our method is not efficient in images which need information of the 
original image strongly. 

Thus there are simply two types in natural images. One is the image which consists 
of a little long edges and large surfaces, and those images depend on a basic and common 
property of natural image strongly. Another is the image which has many short edges and 
small clusters, and it is difficult to distinguish whether the original or degraded images 
in such images. Therefore not only corrupted parts but correct parts may be destroyed 
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by the effect of smoothing in such images. In restoration of images by means of the 
method of statistical mechanics, the effect of smoothing is strong compared with keeping 
the information of original image that the degraded image contains because we use it as 
common property of natural images. Therefore, we need some additional information on 
the original image. 

Furthermore, we found also that the restoration using our method is not affected by 
noise compared with that using the QTsing form. One of the reasons is due to that the 
output, namely the degraded data, takes the close value to input data by the effects of 
noise in our method. This is the most significant property of our method. We expect 
that the restoration using our method gives further better performance when Q-value is 
large. Because we could not distinguish visually the difference between close values with 
increasing the gray-scale level (Q- value), the effect of noise may be suppressed visually. 
Accordingly, we may be obtain the restored image that looks closer to the original image 
intuitively. 

The authors thank Professor Hidetoshi Nishimori for useful discussions. 
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TABLES 

TABLE I. Comparison of performance between our method and the Q-Ising form. The ratios 
of pixels at which the nearest neighboring pixels take the same value in the original image, the 
degraded image and restored image are represented as 'NNP1'. 

TABLE II. Ratio of pixel at which the difference among the value of the nearest neighboring 
pixels is smaller than 2 in the original, degraded and restored images. 

TABLE III. Comparison of performance between using our method and the QTsing form 
for five standard images when ~ 2.0. In this case, the restoration using our method is more 
efficient than that using the Q-Ising form in three standard images: "chair", "girl" and "house". 

TABLE IV. Comparison of performance among using our method, the Q-Ising form and the 
composition for five standard images when ~ 1.0. In this case, the restoration using the 
composition is more efficient than that using our method and the QTsing form in all standard 
images. 
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FIGURES 

FIG. 1. The example of decomposition by means of the method of threshold division in the 
Q = 3 and 6-site case. 

FIG. 2. Magnetization of the original image as a function of the source temperature for 
Q = 3. The solid line corresponds to a globally stable solution mo = 1 and the dotted lines 
correspond to locally stable solutions ttlq = and mo = 1. 

FIG. 3. The magnetization m and corresponding free energy /rs are plotted in the upper-left 
and upper-right figures. The mean square error is plotted in the lower-left figure. The lower-right 
figure represents an enhancement of the mean square error around the optimal values. The solid 
line corresponds to the globally stable solution. 

FIG. 4. Left figure is the mean square error as a function of temperature for several H. 
The solid line corresponds to the optimal value, T m = T s . Right figure represents comparison 
between the mean square error in the bit-decomposed data and the Q-Ising form for the optimal 
hyper-parameters H = H opt . The solid line is that for the BDD case and the dotted line is the 
Q-Ising case. 

FIG. 5. The mean square error in the Q = 4 case is calculated by Monte Carlo simulation 
(the upper figure). Averages over ten samples that (system size is 100 x 100) is taken at each 
data point. The ratio of hyper-parameters H is chosen to be H = 0.6, 0.75 (optimal) and 1.0. 
The mean square error when Q = 8,p = 0.15, ~ 1.0 (the lower figures). The lower left figure 
represents the mean square error in T s = 0.4, and the ratio of hyper-parameters H is chosen to 
be H = 0.6, 0.68 (optimal) and 0.8. The lower right figure represents the mean square error in 
T s = 0.6, and the ratio of hyper-parameters H is chosen to be H = 0.8, 1.0 (optimal) and 1.2. 

FIG. 6. Standard images: "chair", "girl", "house", "lena" and "mandrill". 
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FIG. 7. Standard image "lena". Left side figures represent the restoration using the 
bit-decomposed data, and right side figures represent the restoration using the Q-Ising form 
for flj ~ 1.0. 

FIG. 8. Standard image "lena". Left side figures represent the restoration using the 
bit-decomposed data, and right side figures represent the restoration using the Q-Ising form 
for flj ~ 2.0. 

FIG. 9. Restoration processes using the composition of both our method and the Q-Ising 
form. In the upper process (first process), we decompose a Q- value degraded image into Q — 1 
sets of binary degraded images after we received a (J-value degraded image. Using the binary 
degraded images, we restore the original image. On the other hand, in the lower process (second 
process), we received Q — 1 sets of binary degraded images. After they are translated into a 
Q-value degraded image, we restore the original image using the Q-Ising form. 

FIG. 10. Standard image "lena". The left, center and right figures are the original, degraded 
and restored images, respectively, in the composition process when ifg ~ 1.0. 
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